System for detecting speech interval and recognizing continuous speech in a noisy environment through real-time recognition of call commands

ABSTRACT

The present invention relates to a continuous speech recognition system that is very robust in a noisy environment. In order to recognize continuous speech smoothly in a noisy environment, the system selects call commands, configures a minimum recognition network in token, which consists of the call commands and mute intervals including noises, recognizes the inputted speech continuously in real time, analyzes the reliability of speech recognition continuously and recognizes the continuous speech from a speaker. When a speaker delivers a call command, the system for detecting the speech interval and recognizing continuous speech in a noisy environment through the real-time recognition of call commands measures the reliability of the speech after recognizing the call command, and recognizes the speech from the speaker by transferring the speech interval following the call command to a continuous speech-recognition engine at the moment when the system recognizes the call command.

TECHNICAL FIELD

The present invention relates, in general, to a continuous speechrecognition system, and, more particularly, to a continuous speechrecognition system, which, in order to smoothly perform continuousspeech recognition in noisy environments, selects call commands andconfigures a minimum recognition network composed of a silence intervalaccompanied by noise and the call commands in the form of tokens, sothat real-time speech recognition is continuously performed on inputspeech, the confidence of the input speech is continuously analyzed, andspeech subsequently input by a speaker is recognized, thus being robustin noisy environments.

BACKGROUND ART

The means by which a human being expresses his or her thoughts toanother party can be configured in various forms, but speech is the mostbasic communication of these means used by human beings.

Speech processing by human beings can be divided into the two aspects ofspeech production and speech perception. Speech production refers to aseries of procedures allowing a speaker to communicate his or herintention, and speech perception refers to a series of proceduresenabling spoken contents to be perceived from the speech spoken byanother party. Research into these two types of speech aspects has beenindividually conducted, and has been followed by various academicenvironments such as linguistics, phonetics, phonology, physiology, andanatomy.

Access methods of performing speech recognition from the standpoint ofspeech perception of speech processing can be classified into four typesof methods, that is, an acoustic-phonetic method, a statistical patternrecognition method, an artificial intelligence method, and a neuralnetwork method.

Speech recognition systems using the above-described various accessmethods can be classified into isolated word speech recognition systemsthat recognize isolated words according to the type of vocalization, andcontinuous speech recognition systems that recognize speech made bycontinuously speaking words.

Of these speech recognition systems, an isolated word speech recognitionsystem is a scheme for performing recognition in the sequence of VoiceActivity Detection (VAD)→feature extraction→patterncomparison→recognition, and is a technology which is suitable for smallcomputational loads or small-scale speech recognition, but is unsuitablefor commercialization because unregistered words are rejected only byconfidence detection with the result that errors frequently occur inconfidence detection depending on noisy environments, and because aspecial event such as a recording button for speech recognition isrequired.

FIG. 1 is a diagram briefly showing the construction of a conventionalcontinuous speech recognition network. The continuous speech recognitionnetwork employs a scheme that performs recognition by performingpost-processing on a recognized word sequence using a language model.This scheme is mainly used in large-capacity speech recognition systemsof ten thousand or more words.

A continuous speech recognition system is a scheme for performingrecognition in the sequence of feature extraction→pattern comparison(acoustic model comparison)→language modelcomparison→post-processing→recognition, is suitable for large-scalespeech recognition systems in high-specification server level PersonalComputers (PCs), and also enables a real-time large-scale speechrecognition system to be implemented with the development of storagemedia and computational processing abilities. However, there aredisadvantages because, in order to install such a continuous speechrecognition system in terminals which have gradually become lightweight,still as of yet insufficient storage capacity and complicated floatingcomputation cause problems, and a large computational load and storagecapacity for a large amount of data are required.

FIG. 2 is a diagram showing a speech waveform produced by speaking thesame command in a vehicle which has not been started, and FIG. 3 is adiagram showing a speech waveform produced by speaking the same commandin a vehicle which is traveling with the window open. With regard to thecomparison between FIGS. 2 and 3, the performance of Voice ActivityDetection (VAD) using existing energy or Zero Crossing Rate (ZCR)remarkably decreases due to the influence of the vibrating sound of avehicle engine, the output sound of multimedia devices, and the windwhich is there when the window of the vehicle is open, in an actualvehicle environment. Therefore, in a conventional speech recognitionapparatus in a vehicle, a vehicle driver presses a hot-key to generate aspeech recognition event, and then speech recognition is conducted. Sucha function causes a user to feel inconvenience even if the conventionalspeech recognition apparatus is a small-scale speech recognition systemhaving relatively excellent performance such as the electronic controlof the vehicle or the menu control of a navigation terminal basicallymounted in the vehicle, thus becoming a large obstacle tocommercialization.

DISCLOSURE Technical Problem

The present invention has been made keeping in mind the above problemsoccurring in the prior art, and an object of the present invention is toprovide a system for detecting a speech interval in noisy environmentsusing speech call commands, which is based on the changes of a speechrecognition network and the detection of confidence rate.

Another object of the present invention is to provide a system fordetecting a speech interval in noisy environments using call commands,which can solve the disadvantage of a conventional keyworddetection-based continuous speech recognition technique which cannotperform real-time processing due to the large computational load, alanguage model which must be changed according to the field ofapplication, and the use of post-processing, and can solve thedisadvantage of a conventional isolated word recognition system whichrequires a special event, thus enabling the system to always wait forspeech recognition to occur.

A further object of the present invention is to provide a system fordetecting a speech interval in noisy environments using call commands,which can solve the disadvantage of an isolated word recognition systemwhich requires a special event for speech recognition while solving thedisadvantage of a conventional keyword detection-based continuous speechrecognition technique which requires not only a large computational loadbut also the change of a language model or post-processing, thusenabling the system to always wait for speech recognition to occur.

ADVANTAGEOUS EFFECTS

According to the construction of the present invention, in order tosmoothly perform continuous speech recognition in noisy environments, acall command such as ‘Naraeya’ or ‘Nepiya’ is selected, and a minimumrecognition network composed of silence interval accompanied by noiseand the call command is configured in the form of tokens, so thatreal-time speech recognition is continuously performed on input speech,the confidence of the input speech is continuously analyzed, and thesubsequently input speech intervals of a speaker, as well as bufferedspeech intervals, are applied to a continuous speech recognition engineat the moment at which the call command from the speaker is recognized,thus enabling continuous speech recognition which is very robust innoisy environments.

DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram briefly showing the construction of a conventionalcontinuous speech recognition network;

FIG. 2 is a diagram showing a speech waveform produced by speaking thesame command in a vehicle which has not been started;

FIG. 3 is a diagram showing a speech waveform produced by speaking thesame command while a vehicle is traveling with the window open;

FIG. 4 is a block diagram showing a continuous speech recognition systemincluding a call command search routine in noisy environments accordingto the present invention;

FIG. 5 is a diagram briefly showing the construction of a system fordetecting a speech interval and recognizing continuous speech in noisyenvironments using the real-time recognition of call commands accordingto the present invention;

FIG. 6 is a diagram showing the call command recognition network of thesystem for detecting a speech interval and recognizing continuous speechin noisy environments using the real-time recognition of call commandsaccording to the present invention; and

FIG. 7 is a diagram showing the results of the performance of confidencedetection when the probability at the moment at which the transition tosilence is made corresponds to the uppermost level among theprobabilities of the tokens of a relevant frame in the call commandrecognition network according to the present invention.

BEST MODE

In order to accomplish the above objects, the present invention providesa system for detecting a speech interval and recognizing continuousspeech using real-time recognition of call commands, wherein when aspeaker speaks a call command, the call command is recognized,confidence rate of the call command is measured, and a speech intervalspoken subsequent to the call command is applied to a continuous speechrecognition engine at a moment at which the call command is recognized,thus recognizing speech of the speaker.

In this case, the recognition of the call command is performed by a callcommand recognition network which is implemented using a Left-to-Right(LTR) model, and a speech frame input to the recognition network isconfigured to include predetermined tokens and is compared based onprobability with the recognition network in real time.

In this case, each of the predetermined tokens includes the speech frameand a silence interval accompanied by noise.

In this case, the call command recognition network is configured suchthat, when an accumulated probability of the predetermined token whichis computed in real time after passing through the call commandrecognition network falls within a range of a predetermined upperpercentage, the call command is estimated to have been spoken, and thespeech frame is transferred to a confidence measurement stage.

In this case, the speech frame is configured such that a beam widththereof is limited to 20 or 30 tokens.

In this case, the predetermined upper percentage is 10%.

In this case, the confidence rate is determined by the followingequation:

LLR_(k)(0,λ_(k))=log p(0|λ_(k))−log p(0|λ _(k) )

where LLR is log likelihood ratio, λ_(k) is a phoneme model, and λ _(k)is an anti-phoneme model.

Further, the present invention provides a system for detecting a speechinterval and recognizing continuous speech using real-time recognitionof call commands, comprising a call command recognition networkincluding a call command search routine of searching for a call commandin real time, and a confidence detection routine of detecting confidencerate of the call command found by the call command search routine; and acontinuous speech recognition network for recognizing speech input froma speaker subsequent to the call command at a moment at which the callcommand is recognized by the call command recognition network.

In this case, the call command recognition network comprises an adaptivefilter, a feature extraction unit and a keyword searching unit.

In this case, the continuous speech recognition network comprises atoken passing unit, a keyword spotting unit, a confidence detection unitand a rule check unit.

MODE FOR INVENTION

Hereinafter, techniques, structures and operating characteristics of asystem for detecting a speech interval and recognizing continuous speechin noisy environments using the real-time recognition of call commands(hereinafter referred to as a ‘call command/continuous speechrecognition system’) according to the present invention will bedescribed in detail with reference to the attached drawings.

The call command/continuous speech recognition system according to thepresent invention selects a call command such as ‘Naraeya’ or ‘Nepiya’and configures a minimum recognition network composed of a silenceinterval accompanied by noise and the call command in the form of tokensin order to smoothly perform continuous speech recognition in noisyenvironments, so that real-time speech recognition is continuouslyperformed on input speech, the confidence of the input speech iscontinuously analyzed, and the subsequently input speech intervals of aspeaker, as well as buffered speech intervals, are applied to acontinuous speech recognition engine at the moment at which the callcommand from the speaker is recognized, thus performing continuousspeech recognition which is very robust in noisy environments.

FIG. 4 is a block diagram showing a continuous speech recognition systemincluding a call command search routine in noisy environments accordingto the present invention.

As shown in FIG. 4, the system for detecting a speech interval andrecognizing continuous speech in noisy environments using the real-timerecognition of call commands according to the present invention executesa call command recognition network, which mainly includes a call commandsearch routine for searching for call commands and a confidencedetection routine 421, 422 and 423 for detecting the confidence of thefound call commands, before executing a continuous speech recognitionnetwork 430.

The call command recognition network 420 includes an adaptive filter421, a feature extraction unit 422, and a keyword searching unit 423,and the continuous speech recognition network 430 includes a tokenpassing unit 431, a keyword spotting unit 432, a confidence detectionunit 433, and a rule check unit 444.

In the call command recognition network 420, a minimum recognitionnetwork composed of a silence, interval accompanied by noise and a callcommand from the speaker is configured in the form of tokens, thusenabling real-time speech recognition to be continuously performed oninput speech. When the call command is recognized as the call commandfrom the speaker through the call command search routine, the confidenceof the call command is detected in the confidence detection routinewhich is a subsequent stage. Accordingly, whether the found call commandis an authentic call command intended by the speaker or a false callcommand is determined, thus enabling the recognition of continuousspeech to be performed on commands spoken subsequent to the callcommand.

The construction and function of the call command recognition network420 will be described below.

The adaptive filter 421 is a Finite Impulse Response (FIR) filter forautomatically tracking the spectrum characteristics of surrounding noisefrom noise-containing input speech (all spoken voices including callcommands) and eliminating the surrounding noise.

The feature extraction unit 422 is a component for audibly modeling thespectrum characteristics of the input speech, having passed through theadaptive filter 421, for each frame, and converting the spectrumcharacteristics into robust parameters.

The keyword searching unit 423 determines whether relevant speech is acall command using an acoustic model database (DB), a pronunciationdictionary DB and a language model DB, and then determines whether toexecute the continuous speech recognition routine. The keyword searchingunit 423 includes a token passing unit for configuring a speech frame,which includes a silence interval accompanied by noise and call commandsfrom the speaker, in the form of a minimum number of tokens, and thensearching for call commands, and a confidence detection unit fordetecting the confidence of each recognized call command in relation towhether the recognized call command is an authentic or false command.

When the speech of the speaker is recognized as the call command by thecall command recognition network 420, the process proceeds to thesubsequent stage, that is, the continuous speech recognition network430. The construction and function of the continuous speech recognitionnetwork 430 will be described below.

First, the token passing unit 431 is a component for searching for inputspeech in the configured continuous speech recognition network.

The keyword spotting unit 432 is a component for spotting a keyword froma token for which beam search is conducted using keywords and fillermodels.

The confidence detection unit 433 is a component for determining whethera recognized keyword is authentic or false by detecting the confidenceof the keyword.

The rule check unit 444 is a post-processing component for determiningwhether the results of the recognition correspond to a recognitiontarget sentence.

As shown in FIG. 4, the call command recognition network 420 includesthe search routine and the confidence detection routine for callcommands, thus remarkably reducing the computational load compared tothe conventional continuous speech recognition in which the recognitionnetwork must be executed on all of the continuous speech that is input.

Therefore, the advantages of continuous speech recognition can beapplied even to application fields such as portable devices or embeddedsystems, which have low computational processing ability and a smallstorage capacity.

The call command recognition network 420 includes a component forextracting feature parameters from input speech, and a component forexecuting a modified call command recognition network and detectingconfidence (call command search routine and confidence detectionroutine).

FIG. 5 is a diagram briefly showing the construction of the system fordetecting a speech interval and recognizing continuous speech in noisyenvironments using the real-time recognition of call commands accordingto the present invention.

As shown in FIG. 5, the call command/continuous speech recognitionsystem includes a call command recognition network for recognizing callcommands and detecting the confidence thereof and a continuous speechrecognition network for recognizing speech which is an actual command.The call command/continuous speech recognition system configures arecognition network for call commands (for example, “Naraeya”, “Nepiya”,etc.), executes the recognition network for one call command withrespect to the input speech, and measures the confidence of the callcommand, thus determining whether to start a recognition network foractual continuous speech.

The scheme of the present invention has the advantages of remarkablyreducing the memory capacity and computational processing ability whichare necessary for a continuous speech recognition system using theconventional recognition network.

In order to solve the problems, the following technique is proposed inthe present invention so that a higher rejection rate for unregisteredwords can be obtained and the speech recognition rate can be improvedeven though just a small computational load is used when thekeyword-detection based continuous speech recognition system isimplemented.

FIG. 6 is a configuration diagram showing the call command recognitionnetwork of the system for detecting a speech interval and recognizingcontinuous speech in noisy environments using the real-time recognitionof call commands according to the present invention.

As shown in FIG. 6, the recognition network for call commands isimplemented using a left-to-right model and is configured to enable justthe transition from one silence model to another silence model. On thebasis of probability, every input speech frame is compared with arecognition network for only one word in real time. In this case, thewidth of a beam is limited to only 20 to 30 tokens, thus minimizing thecomputational load.

Since the recognition network is a recognition network for only oneword, a minimum number of (20 to 30) tokens are required. When theaccumulated probability of a relevant token, obtained at the moment atwhich the transition to a silence model is made while real-timecomputation is performed on the frames which are continuously input,corresponds to the top 10% of the 20 to 30 tokens in the call commandrecognition network, it is assumed that an actual call command has beenspoken, and information about a relevant frame is transferred to theconfidence detection routine 421, 422 and 423.

The time-series operating sequence of the call command recognitionnetwork is described below. First, the call command recognition networkis configured (Step 1). The real-time recognition network for a recordedinput speech frame is decoded and is then stored in tokens (Step 2). Inthis case, the individually stored bit values are sorted according tothe log likelihood ratio given by the following Equation,

LLR_(k)(0,λ_(k))=log p(0|λ_(k))  (1)

where LLR_(k) is the log likelihood ratio, λ_(k) is a k-th modelparameter, and p(0|λ_(k)) is the probability of the λ_(k) model of theinput speech frame.

After the accumulated probabilities have been sorted at Step 2, tokenbeam pruning is performed on N (20 to 30) tokens (Step 3). Then, in thefinal stage of the call command recognition network, when theaccumulated probability of a relevant token at the moment at which thetransition to the silence interval is made at Step 3 corresponds to thetop 10% (2 to 3), it is assumed that the call command has been spokenand the process proceeds to the confidence detection routine, otherwisethe process proceeds to Step 2 (Step 4).

Generally, the number of tokens required for the configuration of aspeech recognition network is about the number of words * the number ofwords. That is, the stable number of tokens required in order torecognize continuous speech of about 1,000 words is about 1,000,000.However, in the call command recognition network according to thepresent invention, the accumulated probabilities and model transitionstates for only 20 to 30 tokens are stored. Among these 20 to 30 tokens,only tokens for which the uppermost probability is stored are required.In FIG. 6, when a relevant token is a token for which the accumulatedprobability at the moment at which the transition to silence is made ina₃₃ is stored as the uppermost probability, it is recognized that thecall command has been spoken, and the detection of confidence thereof isperformed. Here, the range of the uppermost probability may be set tothe top 10% range.

The computational load based on such a call command recognition networkis limited to the amount of computation required for the real-timeextraction of Mel Frequency Cepstrum Coefficients (MFCC) parameters andthe decoding of 20 to 30 tokens, and is only an extremely small amountcompared to the computational load based on the continuous speechrecognition network, and thus the computational load can be reducedcompared to the computational load based on the conventional VADdetection technique using energy and ZCR. However, in noisyenvironments, VAD detection rate can be remarkably improved.

When it is determined by the call command recognition network that thecall command has been spoken, the confidence rate of the call command isdetected.

All speech input through a microphone is recognized by one call commandrecognition network, and tokens having excellent accumulated probabilityvalues among the 20 to 30 tokens are transferred to the confidencedetection routine (430 of FIG. 4). Here, in the tokens, information,obtained when the previously input speech makes a transition in the callcommand recognition network (auto segmentation information), is stored.

With respect to input speech (call command) assumed to be a call command‘Naraeya’ or ‘Nepiya’ according to the recognized auto segmentationinformation, a ratio of the probability p(0|λ_(k)) of phoneme models forthe call command recognition network to the probability p(0|λ _(k) ) ofanti-phoneme models, other than the assumed phoneme models, is computed,and thus the confidence rate of the call command is computed. This isrepresented by the following Equation (2),

LLR_(k)(0,λ_(k))=log p(0|λ_(k))−log p(0|λ _(k) )  (2)

where LLR is log likelihood ratio, λ_(k) is a phoneme model, and λ _(k)is an anti-phoneme model.

When LLR is determined by Equation (2) and ‘LLR>threshold’ is satisfied,the continuous speech recognition network is executed, otherwise theinitial state is maintained. Here, the continuous speech recognitiondenotes the recognition of actual commands spoken subsequent to the callcommand.

FIG. 7 is a diagram showing the results of the performance of confidencedetection when the probability at the moment at which the transition tosilence is made corresponds to the uppermost level among theprobabilities of the tokens of a relevant frame in the call commandrecognition network according to the present invention.

As shown in FIG. 7, depicted are the results of the performance ofconfidence detection when the probability at the moment at which thetransition to silence is made corresponds to the uppermost level amongthe probabilities of the tokens of the relevant frame (in the case wherea call command is assumed to be ‘Nepiya’) in the call commandrecognition network. That is, when the input speech frames transferredto the confidence detection routine are backtracked along the storedtokens, the call command is automatically segmented into portions of“N”, “E”, “P”, “IY” and “YA”, as shown in FIG. 7. When the confidence ofeach of the portions is greater than 0 due to a difference between theprobability p(0|λ_(k)) of its own phoneme model and the probabilityp(0|λ _(k) ) of an anti-phoneme model, the call command “Nepiya” isrecognized, and speech spoken subsequent to the call command istransferred to the continuous speech recognition engine to performcontinuous speech recognition. In contrast, when the confidence is lessthan 0, it is determined that the call command is not spoken, and theprocess returns to the initial state.

The speech recognition scheme using the conventional continuous speechrecognition network detects a pause interval of speech using a silencemodel without extracting end points, whereas the call commandrecognition scheme proposed in the present invention executes arecognition network only in a predetermined interval after a callcommand, thus obtaining an advantage in that large-capacity memory andhigh-performance computational processing ability are not required.

Although the preferred embodiments of the present invention have beendisclosed for illustrative purposes, those skilled in the art willappreciate that the present invention can be implemented in otherdetailed forms without changing the technical spirit or essentialfeatures of the present invention. Therefore, those embodiments shouldbe understood to be exemplary rather than restrictive, and the scope ofthe present invention will be defined by the accompanying claims ratherthan the detailed description, and all changes or modifications derivedfrom the meaning and scope of the claims and equivalents thereof shouldbe interpreted as being included in the scope of the present invention.

INDUSTRIAL APPLICABILITY

present invention relates to a speech recognition system, which can beindustrially and widely used in the fields of home networking systems,intelligent robots, vehicle navigation terminals, and speech recognitionsystems for car-PCs.

1. A system for detecting a speech interval and recognizing continuousspeech using real-time recognition of call commands, wherein: when aspeaker speaks a call command, the call command is recognized,confidence rate of the call command is measured, and a speech intervalspoken subsequent to the call command is applied to a continuous speechrecognition engine at a moment at which the call command is recognized,thus recognizing speech of the speaker.
 2. The system according to claim1, wherein: the recognition of the call command is performed by a callcommand recognition network which is implemented using a Left-to-Right(LTR) model; and a speech frame input to the recognition network isconfigured to include predetermined tokens and is compared based onprobability with the recognition network in real time.
 3. The systemaccording to claim 2, wherein each of the predetermined tokens includesthe speech frame and a silence interval accompanied by noise.
 4. Thesystem according to claim 3, wherein the call command recognitionnetwork is configured such that, when an accumulated probability of thepredetermined token which is computed in real time after passing throughthe call command recognition network falls within a range of apredetermined upper percentage, the call command is estimated to havebeen spoken, and the speech frame is transferred to a confidencemeasurement stage.
 5. The system according to claim 4, wherein thespeech frame is configured such that a beam width thereof is limited to20 or 30 tokens.
 6. The system according to claim 4, wherein thepredetermined upper percentage is 10%.
 7. The system according to claim1, wherein the confidence rate is determined by the following equation:LLR_(k)(0,λ_(k))=log p(0|λ_(k))−log p(0|λ _(k) ) where LLR is loglikelihood ratio, λ_(k) is a phoneme model, and λ _(k) is ananti-phoneme model.
 8. A system for detecting a speech interval andrecognizing continuous speech using real-time recognition of callcommands, comprising: a call command recognition network including acall command search routine of searching for a call command in realtime, and a confidence detection routine of detecting confidence rate ofthe call command found by the call command search routine; and acontinuous speech recognition network for recognizing speech input froma speaker subsequent to the call command at a moment at which the callcommand is recognized by the call, command recognition network.
 9. Thesystem according to claim 8, wherein the call command recognitionnetwork comprises an adaptive filter, a feature extraction unit and akeyword searching unit.
 10. The system according to claim 9, wherein thecontinuous speech recognition network comprises a token passing unit, akeyword spotting unit, a confidence detection unit and a rule checkunit.